Dialog control device and method, and robot device

ABSTRACT

A robot can make a dialog customized for the user by first storing various pieces of information appendant to an object as values of the corresponding items of the object. A topic that is related to the topic used in the immediately preceding conversation is then selected. Then, an acquisition conversation for acquiring the value of the item of the selected topic or a utilization conversation for utilizing the value of the item of the topic that is already stored is generated as the next conversation. The value acquired by the acquisition conversation is stored as the value of the corresponding item.

TECHNICAL FIELD

This invention relates to a dialog control device, a dialog controlmethod and a robot device. For example, the present invention cansuitably be used for entertainment robots.

BACKGROUND ART

In recent years, many entertainment robots have been marketed for familyuse. Assume such an entertainment robot lives with a user in a family.When a task is given to the robot by the user, for example a command of“Kick the ball” is given from the user, the robot is required not onlyto perform the task of “kicking the ball” but also to spontaneously takean action that is full of variety in order to avoid one-waycommunications and make everyday interactions with the robot not boringto the user.

Known voice interaction systems are mostly intended to perform a tasksuch as presetting a recording operation of a video recorder, givingguidance for a telephone number or the like (see, for example,Non-Patent Document 1). Other algorithms for generating a response in adialog include simple responsive sentence generating systems includingthat of Eliza, which may be a representative example (see, for example,Non-Patent Document 2).

For an entertainment robot to spontaneously take an action that is fullof variety, data relating to a number of different pieces of knowledgeand action patterns that are full of variety have to be given to therobot in advance for the purpose of various different interactions.However, in reality, the volume of data that can be given to anentertainment robot is limited.

Additionally, when building a robot, it will require tremendous effortsto prepare data that matches the personality of the user of the robot.Therefore, many robots are actually made to have same knowledge inadvance. Then, the user of a robot may hardly feel affinity for therobot and that “the robot is unique to him or her”. On the other hand,each robot that lives in a family is required to take personalized andattractive action.

If it is possible to build a robot who can actively or passively acquirevarious pieces of information such as the name, the birthday, the sex ofthe user, what the user likes and what the user does not like by way ofinteractions with the user so as to be able to communicate with theuser, using the acquired information, such a robot will be free from theabove identified problems and satisfy the user.

Additionally, if such a robot can show the user the learning process ofusing conversations with the user, the user can share the learningexperience. Furthermore, if the robot speaks what the robot has beentaught, the user will have a feeling of intimacy to the robot.

-   Non-Patent Document 1: Information Processing Society, Research    Meeting Report, Voice Language Information Processing 22-8 (1998    Jul. 24), pp. 41-42-   Non-Patent Document 2: Physicality and Computer, Kyoritsu    Publishing, pp. 258-268

However, a number of problems arise when a robot actively or passivelyacquires various pieces of information that are attributes of a personor a thing (object) such as the name, the birthday, and the sex of theuser, what the user likes and what the user does not like, stores themin the form of memory and dialogs, utilizing the memory, as an exampleof conversation of the robot, using information on the user.

Firstly, there is a problem how the robot acquires memory that iscustomized for the user. The memory capacity of a robot is limited andthe framework for storing information is defined in advance so that therobot cannot store everything that appears in conversations.Additionally, it is difficult to process the speech made by the user atunpredictable timings and store them in a memory device for the currentlevel of technology.

However, this problem can be solved by adopting a technique by which therobot makes a speech to acquire memory such as “Tell me the name of thefried of ◯◯!” and discloses what the robot can memorize by acting on theuser so as to have the user tell the value. Thus, the robot can act onthe user so as to be able to collect information with ease when therobot does so to acquire memory.

Secondly, there is also a problem how the robot utilizes the acquiredmemory to make a speech. If the robot makes a speech, randomly utilizingthe acquired memory, transition of topics will occur out of context toembarrass the user. Therefore, there needs a scheme (association) ofutilizing correlated things in the memory the robot has acquired for thenext speech. Additionally, if the acquired memory is output as it is, aspeech can only correspond to an item of the memory on a basis of one toone correspondence so that interactions will be limited in terms ofvariations.

Thirdly, there is a problem when the robot acquires memory and when itutilizes the memory for a conversation. In other words, a state where anaction trying to utilize the memory is taken in a situation where noinformation that is appendant to an object is available or a state wherean action trying to acquire memory is taken in a situation where all theinformation has been acquired needs to be avoided. A scheme needs to beestablished to avoid such a state. Additionally, if the robot memorizesa thing and takes an action immediately thereafter to utilize thememorized thing, the robot may not appear to have a memory device formemorizing things and the action of the robot may not appear asintellectual action. Then, there arises a problem that the action of therobot may not be entertaining.

DISCLOSURE OF THE INVENTION

In view of the above problems, it is therefore an object of the presentinvention to propose a dialog control device, a dialog control methodand a robot device with an enhanced level of entertainment.

According to the present invention, the above object is achieved byproviding a dialog control device comprising: a memory means for storingvarious pieces of information appendant to an object as valuescorresponding to respective items of the object; and a conversationgeneration means for selecting, in response to an item of the objectdefined as topic, another topic relating to the topic used in theimmediately preceding conversation and generating an acquisitionconversation for acquiring the value of the item selected as topic or autilization conversation for utilizing the value of the item in thetopic already stored in the memory means as the next conversation; theconversation generation means being adapted to store the acquired valueacquired by the acquisition conversation as the value of thecorresponding item.

Then, with a dialog control device as defined above, the dialog controldevice can make a conversation that is customized for the user, or theobject, with the user.

In another aspect of the invention, there is provided a dialog controlmethod comprising: a first step of storing various pieces of informationappendant to an object as values corresponding to respective items ofthe object; and a second step of selecting, in response to an item ofthe object defined as topic, another topic relating to the topic used inthe immediately preceding conversation and generating an acquisitionconversation for acquiring the value of the item selected as topic or autilization conversation for utilizing the value of the item in thealready stored topic as the next conversation; the acquired valueacquired by the acquisition conversation being stored in the second stepas the value of the corresponding item.

Then, with a dialog control method as defined above, the dialog controldevice can make a conversation that is customized for the user, or theobject, with the user.

According to the invention, there is provided a robot device comprising:a memory means for storing various pieces of information appendant to anobject as values corresponding to respective items of the object; and aconversation generation means for selecting, in response to an item ofthe object defined as topic, another topic relating to the topic used inthe immediately preceding conversation and generating an acquisitionconversation for acquiring the value of the item selected as topic or autilization conversation for utilizing the value of the item in thetopic already stored in the memory means as the next conversation; theconversation generation means being adapted to store the acquired valueacquired by the acquisition conversation in the memory means as thevalue of the corresponding item.

Then, with a robot device as defined above, the dialog control devicecan make a conversation that is customized for the user, or the object,with the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of an embodiment of robot, showingits functional configuration.

FIG. 2 is a schematic block diagram of the control unit, showing itsconfiguration.

FIG. 3 is a schematic block diagram of the software of the robot,showing its configuration.

FIG. 4 is a conceptual illustration of the format of recording theacquired information.

FIG. 5 is a conceptual illustration of a principal part of the dialogcontrol function of the action control system.

FIG. 6 is a conceptual illustration of the conversation generationsequence.

FIG. 7 is a flow chart of the dialog generation processing sequence.

FIG. 8 is a conceptual illustration of a specific example of processingof the memory acquiring conversation generating section in aconversation generation processing operation.

FIG. 9 is a conceptual illustration of a specific example of processingof the memory acquiring conversation generating section in aconversation generation processing operation.

FIG. 10 is a conceptual illustration of the first specific example ofprocessing of the memory utilizing conversation generating section in aconversation generation processing operation.

FIG. 11 is a conceptual illustration of the first specific example ofprocessing of the memory utilizing conversation generating section in aconversation generation processing operation.

FIG. 12 is a conceptual illustration of the second specific example ofprocessing of the memory utilizing conversation generating section in aconversation generation processing operation.

FIG. 13 is a conceptual illustration of the second specific example ofprocessing of the memory utilizing conversation generating section in aconversation generation processing operation.

FIG. 14 is a conceptual illustration of the third specific example ofprocessing of the memory utilizing conversation generating section in aconversation generation processing operation.

FIG. 15 is a conceptual illustration of the third specific example ofprocessing of the memory utilizing conversation generating section in aconversation generation processing operation.

BEST MODE FOR CARRYING OUT THE INVENTION

Now, an embodiment of the present invention will be described in detailby referring to the accompanying drawings.

(1) Configuration of Robot 1 of this Embodiment (1-1) HardwareConfiguration of Robot 1

FIG. 1 is a schematic block diagram of an embodiment of robot 1, showingits functional configuration. As shown in FIG. 1, the robot 1 comprisesa control unit 2 that comprehensively controls the action of the entirerobot and performs other data processing operations, an input/outputsection 3, a drive section 4 and a power supply section 5.

The input/output section 3 includes CCD (charge coupled device) cameras10 that operate as eyes, microphones 11 that operate as ears and touchsensors 13 that are arranged respectively on the head, the back andother positions to sense any touches made by a user as well as othersensors, which operate as the five senses, or input section.Additionally, it includes as output section a speaker 12 that operatesas mouth and LED indicators (eye lamps) 14, which produce facialexpressions as some of them are combined and turned on, along with otherelements. The output section can express “You feed back(?)” by means ofvoice and flashing lamps in formats other than mechanical motionpatterns of the limbs of the robot 1.

The drive section 4 is a functional block for realizing an action of therobot 1 according to one of the predetermined motion patterns specifiedby the control unit 2. Therefore, it is the object of control in actioncontrol operations. The drive section 4 is a functional module forrealizing certain degree of freedom for each of the joints of the robot1. Each of the joints has a plurality of drive units 15 ₁˜15 _(n)arranged for axes of rolling, pitching, yawing and so on. The driveunits 15 ₁˜15 _(n) respectively include motors 16 ₁˜16 _(n) for givingrise to rotary motions around the respective axes, encoders 17 ₁˜17 _(n)for detecting rotary positions relative to the respective motors 16 ₁˜16_(n) and drivers 18 ₁˜18 _(n) for adaptively controlling the rotarypositions and the rotary speeds of the respective motors 16 ₁˜16 _(n)according to the outputs of the encoders 17 ₁˜17 _(n).

Thus, it is possible to form the robot 1 as a legged robot, which may bea two-legged robot or a four-legged robot, by combining drive units.

The power supply section 5 is a functional module for literallysupplying power to each of the electric circuits in the robot 1. Therobot 1 of this embodiment is of the autonomous drive type and uses abattery. Thus, the power supply section 5 includes a rechargeablebattery 19 and a recharge/discharge control section 20 for controllingthe charging condition of the rechargeable battery 19.

The rechargeable battery 19 is typically a “battery pack” that is acartridge type package of a plurality of lithium ion secondary batterycells.

The recharge/discharge control section 20 grasps the remaining capacityof the rechargeable battery 19 by observing the terminal voltage and therecharging/discharging electric current of the rechargeable battery 19and also the ambient temperature of the rechargeable battery 19 anddetermines the starting time and the terminating time of each rechargingoperation. The starting time and the terminating time of each rechargingoperation that are determined by the recharge/discharge control section20 are notified to the control unit 2 to respectively trigger the startand the end of a recharging operation of the robot 1.

The control unit 2 is the “brain” and mounted in the head section or thetrunk section of the robot 1.

As shown in FIG. 2, the control unit 2 includes a CPU (centralprocessing unit) 21 that operates as main controller and is connected toother circuit components, peripheral devices and a bus. The bus 27 is acommon signal transmission path and includes a data bus, an address busand control bus. Specific addresses (memory addresses or I/O addresses)are assigned to the respective devices on the bus 27. The CPU 21 cancommunicate with any specific device on the bus 27 by addressing it.

The RAM (read access memory) 22 is a volatile and writable memory thatmay be a DRAM (dynamic RAM). It is adapted to load the program codesthat the CPU 21 executes and temporarily store operation data by way ofan active program.

The ROM (read only memory) 23 is a read only memory dedicated topermanently store programs and data. The program codes stored in the ROM23 include a self-diagnostic test program that is executed when power issupplied to the robot 1 and control programs for defining actions of therobot 1.

The control programs of the robot 1 include “sensor input/recognitionprocessing programs” for processing the input to any of the sensors,which may be the CCD cameras 10 or the microphone 11, and recognizing itas symbol, “action control programs” that operate for short term memoryand long term memory and control the actions of the robot 1 on the basisof the sensor input and predetermined action control models and “drivecontrol programs” for controlling the operation of driving each of thejoint motors and the voice output of the speaker 12 according to actioncontrol models.

The nonvolatile memory 24 is an electrically erasable/writable memorydevice such as EEPROM (electrically erasable and programmable ROM) thatis adapted to be used to hold data to be sequentially updated in anon-volatile way. Data to be sequentially updated may include keys ofcryptograph and other security information as well as device controlprograms to be installed after the shipment.

The interface 25 is a device for interconnecting the devices other thanthe control unit 2 so as to allow them exchange data. The interface 25is used to send data to and receive data from the CCD cameras 10, themicrophones 11 and/or the speaker 12 of the input/output section 3. Theinterface 25 is also used to send data to and receive data from thedrivers 18 ₁˜18 _(n) of the drive section 4.

The interface 25 may comprise one or more than one general purposeinterfaces for connecting a computer and peripheral devices includingserial interfaces such as RS (Recommended Standard)-232C, parallelinterfaces such as IEEE (Institute of Electrical and ElectronicsEngineers) 1284, USB (universal serial bus) interfaces, i-Link(IEEE1394) interfaces, SCSI (small computer system interface) interfacesand memory card interfaces (card slots) adapted to accept a PC card or amemory stick so that programs and data may be moved between the controlunit 2 and locally connected external devices.

Alternatively, the interface 25 may include one or more than oneinfrared communication (IrDA) interfaces so that the control unit 2 maycommunicate with external devices wirelessly.

The control unit 2 includes a radio communication interface 26, anetwork interface card (NIC) 28 and so on so that it may communicatewith various external host computers for data communications by way ofshort range radio data communication networks such as Bluetooth, radionetworks such as IEEE802.11b and long range networks such as theInternet.

Due to data communications between the robot 1 and host computers, it ispossible to perform computational operations necessary for controllingcomplex actions of the robot 1 and remotely control the robot 1 by usingremote computer resources.

(1-2) Software Configuration of Robot 1

FIG. 3 is a schematic illustration of the action control system 30 ofthe robot 1 formed by a group of control programs stored in the ROM 23,showing its functional configuration. The robot 1 can be controlled foractions in response to the outcome of recognition of one or more thanone external stimuli and that of the change in the internal conditions.Additionally, the robot 1 can be controlled for actions in response tothe outcome of recognition of one or more than one external stimuli andthat of the change in the internal conditions due to its long termmemory feature that allows it to associate the change in the internalconditions with the external stimuli and memorize it.

The action control system 30 is installed to adopt object-directedprogramming. In this case, each piece of software is handled in terms ofmodule unit that is referred to as “object” in which data and theprocessing procedure for the data are unified. Each object can deliverto and receive data from another object and also invoke another objectby an inter-object communication method that uses message communicationsand common memories.

The action control system 30 includes a visual recognition functionsection 31, an aural recognition function section 32 and a tactilerecognition function section 33 adapted to recognize the externalenvironment on the basis of the outputs of the CCD cameras 10, themicrophones 11 and the touch sensors 13.

The visual recognition function section 31 performs image processingoperations of recognizing faces and colors on the basis of the videosignals output from the CCD cameras 10 and extracts characteristics.Then, the visual recognition function section 31 outputs a face ID(identifier) specific to each person it recognizes, information on theposition and the size of the face image region and information on theposition and the size of each color region and the characteristicamounts obtained as a result of color recognition as well as otherpieces of information.

The aural recognition function section 32 performs sound processingoperations of recognizing voices, speakers and other various sounds onthe basis of the audio signals output from the microphones 11. Then, theaural recognition function section 32 outputs character stringinformation of each word it recognizes and a speaker ID specific to eachspeaker it recognizes as a result of speaker recognition processingperformed on the basis of acoustic characteristics and so on.

The tactile recognition function section 33 recognizes each externalstimulus such as “being stroked” or “being hit” on the basis of thepressure detection signal output from any of the touch sensors 13 andoutputs the outcome of the recognition.

The internal condition management section 34 manages formula models ofemotions of several different types including instincts and sentimentsso as to manage the internal condition of the robot 1 in terms ofinstincts and sentiments in response to each external stimulusrecognized by the visual recognition function section 31, the auralrecognition function section 32 and/or the tactile recognition functionsection 33.

On the other hand, the action control system 30 also includes a shortterm memory section 35 for short term memories that will be lost withtime and a long term memory section 36 that holds information for arelatively long time for the purpose of controlling actions in responseto the outcome of recognition of external stimuli and changes in theinternal conditions. The classification of memory mechanism of shortterm memory and long term memory relies on neuro-psychology.

The short term memory section 35 is a functional module for holding thetargets and the events as recognized by the visual recognition functionsection 31, the aural recognition function section 32 and/or the tactilerecognition function section 33 for a short period of time. For example,it stores the image input from any of the CCD cameras 10 for a shortperiod of time of about 15 seconds.

The long term memory section 36 is used to hold the information that isobtained by learning such as the name of an object for a relatively longperiod of time. The RAM 22 and the non-volatile memory 24 in the controlunit 2 (FIG. 2) are utilized for it.

Actions of the robot 1 that are produced by the action control system 30are globally classified into “reflective actions” that are realized byreflective action section 39, “situation-dependent actions” that arerealized by situation-dependent action layer 38 and “deliberate actions”that are realized by deliberate action layer 37.

The reflective action section 39 is a functional module for realizingreflective acts of the robot in response to external stimuli recognizedby the visual recognition function section 31, the aural recognitionfunction section 32 and/or the tactile recognition function section 33.

A reflective action basically refers to an action of directly receivingthe outcome of recognition of the external information input by any ofthe sensors, classifying it and directly determining the output action.For example, actions such as an action of running after a human face anda nodding action are preferably installed as reflective actions.

The situation-dependent action layer 38 controls actions of the robot 1immediately responding to the current situation of the robot 1 on thebasis of the contents stored in the short term memory section 35 andthose stored in the long term memory section 36 and the internalcondition managed by the internal condition management section 34.

The situation-dependent action layer 38 prepares a state machine foreach action, classifies the outcome of recognition of the externalinformation input from any of the sensors, depending on previous actionsand situations, and demonstrates the action of the robot 1. Thesituation-dependent action layer 38 also realize actions for maintainingthe internal condition within a certain scope (also referred to as“homeostasis actions”) and, when the internal condition goes beyond thespecified scope, it activates the actions of the robot 1 in such a waythat actions for returning the internal condition back to the scope caneasily appear (in reality, actions are selected by taking both theinternal condition and the external environment into consideration), Theresponse time of a situation-dependent action is longer than that of areflective action.

The deliberate action layer 37 plans relatively long term actions on thebasis of the contents of the short term memory section 35 and those ofthe long term memory section 36.

A deliberate action is an action made in a given situation or accordingto an order given by a man on the basis of an assumption or a plan forrealizing the action. For example, an action of searching for a routefrom the current position of the robot and a target position of therobot is a deliberate action. Such an assumption or a plan can requireprocessing time and computational load (processing time in other words)that is longer than the response time that the robot 1 requires tomaintain an interaction, a deliberate action is made on the basis of anassumption or a plan while the robot is making reflective actions andsituation-dependent actions repeatedly.

The deliberate action layer 37, the situation-dependent action layer 38and the reflective action section 39 can be described as higher orderapplication programs that are not dependent on the hardwareconfiguration of the robot 1. On the other hand, hardware-dependentaction control section 40 operates to output voices by way of thespeaker 12 of the input/output section 3 and/or flash the LEDs 14according to a predetermined pattern by driving the corresponding onesof the drive units 15 ₁˜15 _(n) of the drive section 4 according to theorder given from any of the higher order application programs.

(1-3) Configuration of the Internal Condition Management Section 34

Now, the configuration of the internal condition management section 34of the action control system 30 that directly relates to the dialogcontrol function of the robot will be described. The dialog controlfunction of the robot will be described in greater detail hereinafter.

As described above, the internal condition management section 34 managesthe formula models of emotions including instincts and sentiments so asto manage the internal condition of the robot 1 in terms of instinctsand sentiments in response to each external stimulus recognized by thevisual recognition function section 31, the aural recognition functionsection 32 and/or the tactile recognition function section 33.

For the purpose of the invention, there are provided nine instinctiveelements including fatigue, body temperature, pain, hunger, thirst,affection, curiosity, elimination and sexual desire and correspondinglyeighteen emotional elements including happiness, sadness, anger,surprise, disgust, fear, frustration, boredom, somnolence,gregariousness, patience, tension, relaxation, alertness, guilt, spite,loyalty, submission and jealousy.

The internal condition management section 34 has parameters forexpressing the intensity of each of the instinctive elements and theemotional elements. The instincts and the sentiments of the robot 1 areincessantly changed as the parameters are periodically updated on thebasis of the outcome of recognition of the visual recognition functionsection 31, the aural recognition function section 32 and/or the tactilerecognition function section 33, the elapsed time and so on.

More specifically, if the variation of each instinctive element ascomputationally determined by means of a predetermined operationalformula on the basis of the outcome of recognition of the visualrecognition function section 31, the aural recognition function section32 and/or the tactile recognition function section 33, the elapsed timesince the last update and so on is ΔI [k], the current parameter valueof the instinctive element is I [k] and the coefficient for expressingthe sensitivity of the instinctive element is ki, the internal conditionmanagement section 34 computes with a predetermined cycle period theparameter value I [k+1] of the next cycle period, using the formula ofI[k+1]=I[k]+k1×ΔI[k]  (1)and updates the parameter value of the instinctive element by replacingthe current parameter value of the instinctive element with the outcomeof the computation.

Additionally, if the variation of each emotional element ascomputationally determined by means of a predetermined operationalformula on the basis of the outcome of recognition of the visualrecognition function section 31, the aural recognition function section32 and/or the tactile recognition function section 33, the elapsed timesince the last update and so on is ΔE [t], the current parameter valueof the emotional element is E [t] and the coefficient for expressing thesensitivity of the emotional element is ke, the internal conditionmanagement section 34 computes with the predetermined cycle period theparameter value E [t+1] of the next cycle period, using the formula ofE[t+1]=E[t]+ke×ΔE[t]  (2)and updates the parameter value of the emotional element by replacingthe current parameter value of the emotional element with the outcome ofthe computation.

Note that the extent to which the outcome of recognition of the visualrecognition function section 31, the aural recognition function section32 and/or the tactile recognition function section 33 and so on affectseach of the instinctive elements and the emotional elements ispredefined. For example, the outcome of recognition of “being stroked”by the tactile recognition function section 33 most greatly affects thevariation ΔI [k] of the parameter value of “affection” out of theinstinctive elements and the variation ΔE [t] of the parameter value of“happiness” out of the emotional elements.

(2) Dialog Control Function of Robot 1 (2-1) Dialog Control Function ofRobot 1

Now, the dialog control function with which the robot 1 is provided willbe described below.

The robot 1 is provided with a dialog control function of acquiringinformation on each of a certain number of predetermined items (to bereferred to as the value of the item hereinafter) including the name,the birthday, the favorites and so on of the user or some other object(to be referred to as the user or the like hereinafter) and stores itwith a predetermined format as shown in FIG. 4 so as to be able todialog with the user, utilizing the stored value of each item.

In FIG. 4, the columns show the values assigned to the respective items(“name”, “kind”, “face ID”, “speaker ID”, “birthday”, “favorite” and“friend”) and the rows show the values acquired for the respective itemsof each object (“Yukiko”, “human”, “0”, “1”, “73/5/2”, “tea” and“Kazumi”). The first numeral of each row denotes the ID of the object(object ID) assigned to the object when the object is registered for thefirst time.

While all the items of each of the object have already acquiredrespective values in FIG. 4, the values of “name”, “speaker ID”,“birthday”, “favorite” and “friend” are acquired by way of various voicerecognition processing operations of the aural recognition functionsection 32 in dialogs with the user and stored, while the values of“face ID” and “kind” are acquired by way of various image recognitionprocessing operations of the visual recognition function section 31 indialogs with the user and stored.

In FIG. 4, the numeral arranged to the left of each value indicates thedegree of impression relative to the value. The degree of impressionprovides a measure for determining if the robot 1 may use the item, orthe topic, in conversations in the future or not. In the case of FIG. 4,a higher degree of impression may be rated as good (would like to use itin the next conversation), whereas a lower degree of impression may berated as bad (would not like to use it in conversations). In thisembodiment, the degree of impression, each degree of impression isobtained by determining the difference between the parameter value of“affection” immediately before acquiring the value of the related itemand the parameter value of “affection” immediately after acquiring thevalue of the related item in the internal condition management section34.

The dialog control function is mainly realized by way of processingoperations of the situation-dependent action layer 38 of the actioncontrol system 30 as described above by referring to FIG. 3. As shown inFIG. 5, from the viewpoint of processing operations relating to thedialog control function, the situation-dependent action layer 38 can befunctionally divided into a memory acquiring conversation generatingsection 50 that generates conversations for acquiring values of theitems including those of the user (to be referred to as acquiringconversations hereinafter), a memory utilizing conversation generatingsection 51 that generates conversations for utilizing the acquiredvalues of the items including those of the user (to be referred to asutilizing conversations hereinafter) and a situation judging section 52that controls the memory acquiring conversation generating section 50and the memory utilizing conversation generating section 51.

As the situation judging section 52 recognizes the presence of the userwith whom the robot 1 can dialog on the basis of the outcome ofrecognition of the visual recognition function section 31 and theoutcome of recognition of the aural recognition function section 32 thatcan be obtained by way of the short term memory section 35 (FIG. 3), itsends the face ID and the speaker ID of the user acquired as the outcomeof recognition of the visual recognition function section 31 and that ofthe aural recognition function section 32 to the memory acquiringconversation generating section 50 and the memory utilizing conversationgenerating section 51 and, at the same time, enquires the memoryacquiring conversation generating section 50 and the memory utilizingconversation generating section 51 if they can respectively generate anacquiring conversation or a utilizing conversation or not (Step SP1A,Step SP1B).

The memory acquiring conversation generating section 50 and the memoryutilizing conversation generating section 51 perform conversationgenerating processing operations of generating a topic and determiningthe method of utilizing the topic in order to respectively generate anacquiring conversation and a utilizing conversation according to thepredetermined rules for generating a topic and those for determining themethod of utilizing the topic (Step SP2A, Step SP2B).

A total of six rules are provided in this embodiment as rules forgenerating a topic.

The first topic generation rule is a rule of selecting an item of thesame object that is different from the topic used in the immediatelypreceding conversation as the next topic. In this embodiment, the itemhaving the highest degree of impression is selected first. For example,referring to FIG. 4, if the topic of the immediately precedingconversation is the “birthday” of the “object ID1”, the item having thehighest degree of impression is selected as the topic of the nextconversation out of all the times of the same “object ID1”.

The second topic generation rule is a rule of selecting an item relatingto the same object used in the immediately preceding conversation as thenext topic. For example, referring to FIG. 4, if the topic of theimmediately preceding conversation is the “favorite” of the “objectID1”, the “dislike” of the “object ID1” is selected as the topic of thenext conversation.

The third topic generation rule is a rule of selecting one of the itemsof the object that can be identified from the value of the item, or thetopic, that is used in the immediately preceding conversation. In thisembodiment, the item having the highest degree of impression is selectedfirst. For example, referring to FIG. 4, if the topic of the immediatelypreceding conversation is the “friend” of the “object ID1”, one of theitems of the “fiend”, who is the “object ID2”, is selected as the topicof the next conversation.

The fourth topic generation rule is a rule of selecting the item of thesame object that is same as the topic of the immediately precedingconversation for the next topic. For example, referring to FIG. 4, ifthe topic of the immediately preceding conversation is the “birthday” ofthe “object ID1”, the same “birthday” of the same “object ID1” isselected as the topic of the next conversation.

The fifth topic generation rule is a rule of selecting a topic having avalue of some other object same as that of the item, or the topic, ofthe immediately preceding conversation is selected as the next topic.For example, referring to FIG. 4, if the topic of the immediatelypreceding conversation is the “favorite” of the “object ID1”, the“favorite” of the “object ID3”, which is “tea” and having a value sameas that of the “favorite” of the “object ID1”, is selected as the nexttopic.

The sixth topic generation rule is a rule of selecting an item of someother object having a value relating to that of the item, or the topic,used in the immediately preceding conversation as the next topic. Forexample, referring to FIG. 4, if the topic of the immediately precedingconversation is the “favorite” of the “object ID1”, the favorite of the“object ID6”, which is the “horse race”, is selected from the “favorite”of “horse”. Thus, the “favorite” of the “object ID6” is selected as thetopic of the next conversation.

Note that, out of the first through sixth topic generation rules, thefirst through third topic generation rules can be utilized for bothgenerating acquiring conversations and generating utilizingconversations, whereas the third through sixth rules can be used forgenerating utilizing conversations, although they cannot be used forgenerating acquiring conversations.

Thus, the memory acquiring conversation generating section 50 randomlyselects one of the first through third topic generation rules, while thememory utilizing conversation generating section 51 randomly selects oneof the first through sixth topic generation rules. Then, if the topic ofthe immediately preceding conversation is “the favorite of the objectID1”, “the friend of the object ID1”, “the dislike of the object ID1”,“the birthday of the object ID2”, “the birthday of the object ID1”, “thefavorite of the object ID3” or the “favorite of the object ID6” will beselected as the next topic according to the selected topic generationrules.

On the other hand, a total of three rules are provided in thisembodiment for determining the method of utilizing the topic.

The first topic utilizing method determining rule is a rule ofgenerating a speech by utilizing the value of the corresponding itemstored in the long term memory section 36. If, for example, “thefavorite of the object ID1” is generated as topic, a speech saying “Teais Yukiko's favorite, isn't it?” or “What is Yukiko's favorite?” will begenerated.

The second topic utilizing method determining rule is a rule ofretrieving a data base on the basis of the value of a corresponding itemstored in the long term memory section 36 and reading a related item soas to generate a speech by utilizing the read out item. If, for example,“the birthday of the object ID1” is generated as topic, the data base isretrieved by using the value “73/5/2” of the “birthday” as keyword and aspeech saying “May 2nd is the day of traffic advertisement isn't it” isgenerated by utilizing the retrieved item of “the day of trafficadvertisement”.

The third topic utilizing method determining rule is a rule ofretrieving a data base, using a related value that can be obtained fromthe value of a corresponding item stored in the long term memory 36 bycomputations or by association, and reading a further related item togenerate a speech by utilizing the read out item. If, for example, “thebirthday of the object ID1” is generated as topic, the data base isretrieved by using a horoscope data base, using “Taurus” obtained bycomputations from “73/5/2”, which is the “birthday”, as keyword, and aspeech saying “People of Taurus are patient aren't they?” is generatedby utilizing the obtained item of “patient”.

Of the first through third topic utilizing method determining rules, thefirst topic utilizing method determining rule can be used for bothgenerating acquiring conversations and generating utilizingconversations, whereas the second and third topic utilizing methoddetermining rules can be used for generating utilizing conversations,although they cannot be used for generating acquiring conversations.

Thus, the memory acquiring conversation generating section 50 selectsthe first topic utilizing method determining rule, while the memoryutilizing conversation generating section 51 randomly selects one of thefirst through third topic utilizing method determining rules. Then, theygenerate a speech for acquiring information or a speech utilizing theacquired information according to the selected topic utilizing methoddetermining rules.

When the memory acquiring conversation generating section 50 and thememory utilizing conversation generating section 51 succeed ingenerating a topic and determining a method of utilizing the topic as aresult of topic generation processing, they notify the situation judgingsection 52 of the success as well as the topic and the method ofutilizing the topic. When, on the other hand, they do not succeeded ingenerating a topic and determining a method of utilizing the topic, theynotify the situation judging section 52 of the failure (Step SP3A, StepSP3B).

On the other hand, if the situation judging section 52 receives anotification of the success in generating a topic and determining amethod of utilizing the topic from both the memory acquiringconversation generating section 50 and the memory utilizing conversationgenerating section 51, it selects either the memory acquiringconversation generating section 50 and the memory utilizing conversationgenerating section 51 on the basis of the first extent as determined bythe number of items for which values are not acquired yet out of all theitems of the partner of dialog and the second extent as determined bythe number of items for which values are already acquired out of all theitems of the partner of dialog (Step SP4).

More specifically, the situation judging section 52 selects the memoryacquiring conversation generating section 50 when the first extent isgreater than the second extent, whereas the section 52 selects thememory utilizing conversation generating section 51 when the firstextent is smaller than the second extent.

Then, as the situation judging section 52 selects either the memoryacquiring conversation generating section 50 or the memory utilizingconversation generating section 51, it issues a command for starting aprocessing operation of acquiring a conversation or utilizing aconversation (to be referred to as conversation generating processingoperation) on the basis of the topic and the method of utilizing thetopic to the memory acquiring conversation generating section 50 or thememory utilizing conversation generating section 51, whicheverappropriate (Step SP5).

If, on the other hand, the situation judging section 52 receives anotification of the success in generating a topic and determining amethod of utilizing the topic from either the memory acquiringconversation generating section 50 or the memory utilizing conversationgenerating section 51, it issues a command for starting a conversationgenerating processing operation on the basis of the topic and the methodof utilizing the topic to the memory acquiring conversation generatingsection 50 or the memory utilizing conversation generating section 51,whichever appropriate (Step SP5).

Then, the memory acquiring conversation generating section 50 or thememory utilizing conversation generating section 51, to whom the commandis issued, starts a conversation generating processing operation.Firstly, it accesses the internal condition management section 34 andacquires the parameter value of “affection” out of the instinctiveelements. Then, it sequentially sends out character string data D1 ofthe contents of each of a series of speeches for an acquiringconversation or a utilizing conversation that include the speeches foracquiring information (to be referred to as acquiring speecheshereinafter) or speeches for utilizing the acquired information (to bereferred to as utilizing speeches hereinafter) that are generated on thebasis of the topic and the method of utilizing it as determined in theabove described manner to voice synthesizing section 53 of thehardware-dependent action control section 40.

Thus, as a result, a voice signal S1 is generated by the voicesynthesizing section 53 on the basis of the character string data D1 andgiven to the speaker 12 (FIG. 1) so that, for example, the sound of anacquiring conversation of a series of speeches including an acquiringspeech saying “What is Yukiko's favorite” or the sound of a utilizingconversation of a series of speeches including a utilizing speech saying“Tea is Yukiko's favorite, isn't it?” is output from the speaker 12(Step SP6).

At this time, the response of the user to the topic is collected by themicrophones 11 and applied to the aural recognition function section 32of the action control system 30 (FIG. 3) and the sounded voice isrecognized by the aural recognition function section 32.

If the memory acquiring conversation generating section 50 is selectedin Step SP4, the memory acquiring conversation generating section 50extracts an answer to the question (in other words, the value of theitem to be acquired at the time) from the response of the user to theacquiring speech on the basis of the acquiring speech as a result of thevoice recognition of the aural recognition function section 32 andstores it in the long term memory section 36 typically in the format asdescribed above by referring to FIG. 4 (Step SP6).

Further, the memory acquiring conversation generating section 50accesses the internal condition management section 34 to acquire theparameter value of “affection” at this time and compute the differencebetween the acquired parameter value and the parameter value of“affection” acquired immediately before the conversation. Then, thememory acquiring conversation generating section 50 stores the outcomein the long term memory 36 as the values of items of the user and so onacquired in the above manner as the degree of impression for that value(Step SP6).

Meanwhile, as the acquiring conversation or the utilizing conversationends, the memory acquiring conversation generating section 50 or thememory utilizing conversation generating section 51, whicheverappropriate, sends out a notification of the end of the conversation tothe situation judging section 52 (Step SP7). Then, upon receiving thenotification, the situation judging section 52 notifies the topic andthe method of utilizing the topic as notified in Step SP3A or Step SP3Bfrom the memory acquiring conversation generating section 50 or thememory utilizing conversation generating section 51 it selected in StepSP4 to the memory acquiring conversation generating section 50 and thememory utilizing conversation generating section 51 (Step SP8A, StepSP8B).

Thus, each of the memory acquiring conversation generating section 50and the memory utilizing conversation generating section 51 stores thetopic and the method of utilizing, of which they are notified, asconversation history (Step SP9A, Step SP9B) and go on sequentially andrespectively generating acquiring conversations and utilizingconversations so as not to use the same topic in a single dialog withthe same dialog partner in a manner as described above until theconversation ends (Step SP10A-Step SP1A˜Step SP10A, Step S10B-StepSP1B˜Step SP10B).

In this way, the robot 1 is adapted to carry on a conversation on topicsspecifically designed for the user, transitionally switching topics fromone to another in a natural way, by sequentially acquiring and utilizingvarious pieces of information on the user.

(2-2) Specific Processing Operations of Each of the ConversationGenerating Sections for Generating a Topic

Now, specific processing operations of the memory acquiring conversationgenerating section 50 and those of the memory utilizing conversationgenerating section 51 for generating a topic will be described below.

Each of the memory acquiring conversation generating section 50 and thememory utilizing conversation generating section 51 generates a topicand a method of utilizing the topic, following the topic generatingprocessing sequence RT1 shown in FIG. 7.

When an inquiry if an acquiring conversation and a utilizingconversation can be generated or not arrives from the situation judgingsection 52 in Step SP1A and in Step SP1B of FIG. 6, each of the memoryacquiring conversation generating section 50 and the memory utilizingconversation generating section 51 starts the topic generatingprocessing sequence RT1 in Step SP20 and then randomly selects one ofthe first through third topic generation rules or the first throughsixth topic generation rules, whichever appropriate, in the subsequentstep, or Step SP21.

Then, each of the memory acquiring conversation generating section 50and the memory utilizing conversation generating section 51 proceeds toStep SP22, where it generates a topic according to the topic utilizationrule selected in Step SP1. Thus, if the topic of the immediatelypreceding conversation is “the favorite of the object ID1”, thegenerated topic will be “the friend of the object ID1”, “the dislike ofthe object ID1” or the like.

Subsequently, each of the memory acquiring conversation generatingsection 50 and the memory utilizing conversation generating section 51proceeds to Step SP23, where it reads out the data in the memory areathat corresponds to the “name” of the object ID1 and the data in thememory area that corresponds to the corresponding item of the object ID1in the long term memory section 36. If, for example, the topic selectedin Step SP21 is “the friend of the object ID1”, the data in the memoryarea of the long term memory section 36 that corresponds to the “name”column of the “object ID1” in FIG. 4 and the data in the memory area ofthe long term memory section 36 that corresponds to the “friend” columnof the “object ID1” are read out.

Note, however, there may be cases where no value is acquired yet for the“name” of the object ID1 and for the corresponding item of the objectID1, and no data is stored in the read out memory area.

Thus, each of the memory acquiring conversation generating section 50and the memory utilizing conversation generating section 51 judges inthe next step, or Step SP24, if it can generate an acquiringconversation or a utilizing conversation, whichever appropriate, or not,or a value is acquired for each of the desired items or not on the basisof the data read out in Step SP23.

If the outcome of Step SP24 is negative, the memory acquiringconversation generating section 50 or the memory utilizing conversationgenerating section 51, whichever appropriate, returns to Step SP21 andrepeats the steps of Step SP21 through Step SP24 until a positive answeris obtained in Step SP24.

If, on the other hand, the outcome of Step SP24 is positive, the memoryacquiring conversation generating section 50 or the memory utilizingconversation generating section 51, whichever appropriate, proceeds toStep SP25. If the conversation generating section is the memoryacquiring conversation generating section 50, it selects the first topicutilizing method determining rule as topic utilizing method determiningrule in Step SP25. If, on the other hand, the conversation generatingsection is the memory utilizing conversation generating section 51, itrandomly selects one of the first through third topic utilizing methoddetermining rules as topic utilizing method determining rule in StepSP25.

Then, each of the memory acquiring conversation generating section 50and the memory utilizing conversation generating section 51 proceeds toStep SP26, where it judges if the combination of the topic and themethod of utilizing it as selected respectively in Step SP21 and StepSP25 is already used in a conversation between the robot 1 and the useror not by checking the stored conversation history.

If the memory acquiring conversation generating section 50 or the memoryutilizing conversation generating section 51, whichever appropriate,obtains a negative outcome in Step SP26, it proceeds to Step SP27, whereit terminates the topic generating processing sequence RT1 andsubsequently notifies the situation judging section 52 of the success ingenerating a topic and determining a method of utilizing the topic (StepSP3A, Step SP3B of FIG. 6).

If, on the other hand, the memory acquiring conversation generatingsection 50 or the memory utilizing conversation generating section 51,whichever appropriate, obtains a positive outcome in Step SP26, itreturns to Step SP25 and selects a new top utilizing method determiningrule and repeats the loop of Steps SP25-SP26-SP25 until all thecombinations of the topic utilizing method determining rules are testedfor the topic or a positive outcome is obtained in Step SP26.

If the memory acquiring conversation generating section 50 or the memoryutilizing conversation generating section 51, whichever appropriate,tests all the combinations of the topic utilizing method determiningrules for the topic but no negative outcome is obtained in Step SP27 asa result of the processing to confirm that there is no topic utilizingmethod determining rule that can be selected for the topic, it returnsto Step SP21 to generate a new topic and repeats the above stepsstarting from Step SP2.

Additionally, if the memory acquiring conversation generating section 50or the memory utilizing conversation generating section 51, whicheverappropriate, confirms that it is not possible to generate a conversationby any one of the combinations of all the topics and all the utilizingmethods (not able to get to Step SP27) by repeating the processing ofStep SP21 through SP26, it proceeds to Step SP28, where it terminatesthe topic generating processing sequence RT1 and notifies the situationjudging section 52 of the fact that it cannot generate any topic nor atopic utilizing method (Step SP3A, Step SP3B of FIG. 6).

In this way, the memory acquiring conversation generating section 50 andthe memory utilizing conversation generating section 51 can generate atopic and determine a method of utilizing the topic.

(2-3) Specific Processing Operations of Each of the ConversationGenerating Sections for Generating a Conversation

Now, specific processing operations of the memory acquiring conversationgenerating section 50 and those of the memory utilizing conversationgenerating section 51 for generating a conversation will be describedbelow.

Each of the memory acquiring conversation generating section 50 and thememory utilizing conversation generating section 51 has templates asshown in FIG. 8 to be used for generating an acquiring speech foracquiring the value of each of the items as illustrated in FIG. 4 and autilizing speech for utilizing the value of the item.

The memory acquiring conversation generating section 50 and the memoryutilizing conversation generating section 51 are so adapted that, whengenerating conversations, they respectively generate character stringdata D1 of an acquiring speech for acquiring the value of thecorresponding item and character string data D1 of a utilizing speechfor utilizing the value of the item by applying, for example, the valueof the “name” and that of the “favorite” of the object that provides thecurrent topic to the corresponding part of the corresponding template ofthe item.

Additionally, the memory acquiring conversation generating section 50and the memory utilizing conversation generating section 51 have a statetransition model that predefines what speech is to be made at whattiming for each utilizing method, or for each of the above describedfirst through third topic utilizing method determining rules in order torespectively generate an acquiring conversation of a series of speechesincluding an acquiring speech and a utilizing conversation of a seriesof speeches including a utilizing speech generated on the basis of thetopic and the method of utilizing the topic as determined in Step SP2Aand Step SP2B of FIG. 6.

In a state transition model, nodes (Node 1 through Node 6) thatrepresent respective states are linked by paths, which are also referredto as arcs (Ac1 through Ac6), and a speech to be made and a processingoperation to be performed are correlated to each of the paths as shownin FIG. 9.

When generating conversations including an acquiring conversation and autilizing conversation, the memory acquiring conversation generatingsection 50 and the memory utilizing conversation generating section 51make the states (Node 1 through Node 6) sequentially transit accordingto the respective corresponding state transition models and sequentiallysend out the character string data D1 correlated to the path (arcs Ar1through Ar7) to be passed through to the voice synthesizing section 53(FIG. 5) so as to sequentially output the predetermined voiced speechesin a predetermined order from the speaker 12.

At this time, the acquiring speech and the utilizing speech arecorrelated to a path of the respective state transition models so thatthe memory acquiring conversation generating section 50 and the memoryutilizing conversation generating section 51 respectively generate theacquiring speech and the utilizing speech in a manner as described abovewhen passing through the respective paths and send them out thecharacter string data D1 to the voice synthesizing section 53 so as tooutput the acquiring speech and the utilizing speech from the speaker12.

In this way, the memory acquiring conversation generating section 50 andthe memory utilizing conversation generating section 51 are adapted torespectively generate an acquiring conversation and a utilizingconversation containing respective series of speeches includingrespectively an acquiring speech and a utilizing speech that match thetopic and the method of utilizing the topic as determined in Step SP2Aand Step SP2B in FIG. 6.

FIGS. 8 and 9 illustrate a specific example of conversation (acquiringconversation) that may be held between the robot and the user in orderto acquire the value of an item under the control of the memoryacquiring conversation generating section 50. In the illustratedexample, assume that the topic is “the favorite of the object ID1” andthe first topic utilizing method determining rule is selected as methodof utilizing the topic.

Then, the memory acquiring conversation generating section 50 initiallymakes a transition of state from Node 1 to Node 2 and outputs characterstring data D1 of a speech of “May I ask about Yukiko?” that iscorrelated to arc Ac1 to the voice synthesizing section 53 in order toask the user for permission to acquire a memory and also access theinternal condition management section 34 (FIG. 5) in order to acquirethe parameter value of “affection”.

Then, if the memory acquiring conversation generating section 50confirms at Node 2 that the user responds positively, saying “Yes”, onthe basis of the outcome of the voice recognition of the auralrecognition function section 32 (FIG. 5), it makes a transition of statefrom Node 2 to Node 3 and outputs character string data D1 of theacquiring speech correlated to arc Ac2, the speech of “What is thefavorite of Yukiko?” in this example, to the voice synthesizing section53.

If, on the other hand, the memory acquiring conversation generatingsection 50 confirms at Node 2 that user responds negatively, saying“No”, on the basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 2 to Node 6 and outputs character string data D1 of the acquiringspeech correlated to arc Ac6, the speech of “Sorry” in this example, tothe voice synthesizing section 53.

Then, if the memory acquiring conversation generating section 50confirms at Node 3 that the user responds, saying “My favorite is tea.”,on the basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 3 to Node 6 and outputs character string data D1 of the acquiringspeech correlated to arc Ac3, the speech of “Tea, right?” in thisexample, to the voice synthesizing section 53.

Subsequently, if the memory acquiring conversation generating section 50confirms at Node 4 that the user responds negatively, saying “No”, onthe basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 4 to Node 3 and outputs character string data D1 of the speechcorrelated to arc Ac7, the speech of “What is the favorite of Yukiko?”in this example, to the voice synthesizing section 53.

If, on the other hand, the memory acquiring conversation generatingsection 50 confirms at Node 4 that the user responds positively, saying“Yes”, on the basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 4 to Node 5 and outputs character string data D1 of the speechcorrelated to arc Ac4, the speech of “I got it.” in this example, to thevoice synthesizing section 53. It also accesses the internal conditionmanagement section 34 (FIG. 5) to acquire the parameter value of“affection” and compute the difference between the acquired parametervalue of “affection” and the parameter value of “affection acquiredbefore outputting the acquiring speech.

Additionally, the memory acquiring conversation generating section 50subsequently makes a transition of state from Node 5 to Node 6 andstores “tea” as the value of the item of “favorite” of the user asacquired in a manner as described above in the long term memory 36 (FIG.5) and, at the same time, the difference of the parameter values of“affection” as computed in a manner as described above in the long termmemory 36 as the degree of impression for that value.

As the memory acquiring conversation generating section 50 makes atransition of state to Node 6, it notifies the situation judging section52 (FIG. 5) of the fact that a series of speeches is over andsubsequently ends the processing operation for generating this series ofacquiring speeches.

Note that only the first topic utilizing method determining rule can beused for an acquiring conversation. In other words, all acquiringconversations are to be held according to the above described singlestate transition model.

FIGS. 10 and 11 illustrate a first specific example of conversation(utilizing conversation) that may be held between the robot and the userin order to utilize the value of an item under the control of the memoryutilizing conversation generating section 51. In the illustratedexample, assume that the topic is “the birthday of the object ID1” andthe first topic utilizing method determining rule is selected as methodof utilizing the topic.

Then, the memory utilizing conversation generating section 51 initiallymakes a transition of state from Node 10 to Node 11 and outputscharacter string data D1 of a speech of “May I ask about Yukiko?” thatis correlated to arc Ac10 to the voice synthesizing section 53 in orderto ask the user for permission to talk about a topic relating to theuser and also access the internal condition management section 34 (FIG.5) in order to acquire the parameter value of “affection”.

Then, if the memory utilizing conversation generating section 51confirms at Node 11 that the user responds positively, saying “Yes”, onthe basis of the outcome of the voice recognition of the auralrecognition function section 32 (FIG. 5), it makes a transition of statefrom Node 11 to Node 12 and outputs character string data D1 of theutilizing speech correlated to arc Ac11, the speech of “Yukiko'sbirthday is 73/5/2, isn't it?” in this example, to the voicesynthesizing section 53. Thereafter, the memory utilizing conversationgenerating section 51 further makes transition of state from Node 12 toNode 13.

If, on the other hand, the memory utilizing conversation generatingsection 51 confirms at Node 11 that user responds negatively, saying“No”, on the basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 11 to Node 13 and outputs character string data D1 of the speechcorrelated to arc Ac13, the speech of “Sorry” in this example, to thevoice synthesizing section 53.

As the memory utilizing conversation generating section 51 makes atransition of state to Node 13, it notifies the situation judgingsection 52 (FIG. 5) of the fact that a series of speeches is over andsubsequently ends the processing operation for generating this series ofutilizing speeches.

FIGS. 12 and 13 illustrate a second specific example of conversation(utilizing conversation) that may be held between the robot and the userin order to utilize the value of an item under the control of the memoryutilizing conversation generating section 51. In the illustratedexample, assume that the topic is “the birthday of the object ID1” andthe second topic utilizing method determining method is selected asmethod of utilizing the topic.

Then, the memory utilizing conversation generating section 51 initiallymakes a transition of state from Node 20 to Node 21 and outputscharacter string data D1 of a speech of “May I ask about Yukiko?” thatis correlated to arc Ac20 to the voice synthesizing section 53 in orderto ask the user for permission to talk about a topic relating to theuser and also access the internal condition management section 34 (FIG.5) in order to acquire the parameter value of “affection”.

Then, if the memory utilizing conversation generating section 51confirms at Node 21 that the user responds positively, saying “Yes”, onthe basis of the outcome of the voice recognition of the auralrecognition function section 32 (FIG. 5), it makes a transition of statefrom Node 21 to Node 22 and reads the value of the “birthday” of “theobject ID1” and retrieves the memorial day data base, using the obtainedvalue of “73/5/2”. As the memory utilizing conversation generatingsection 51 acquires “the day of traffic advertisement”, it outputscharacter string data D1 of the speech of, for example, “May 2nd is theday of traffic advertisement, you know.”, to the voice synthesizingsection 53. Thereafter, the memory utilizing conversation generatingsection 51 further makes transition of state from Node 22 to Node 23.

If, on the other hand, the memory utilizing conversation generatingsection 51 confirms at Node 21 that user responds negatively, saying“No”, on the basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 21 to Node 23 and outputs character string data D1 of the speechcorrelated to arc Ac23, the speech of “Sorry” in this example, to thevoice synthesizing section 53.

As the memory utilizing conversation generating section 51 makes atransition of state to Node 23, it notifies the situation judgingsection 52 (FIG. 5) of the fact that a series of speeches is over andsubsequently ends the processing operation for generating this series ofutilizing speeches.

FIGS. 14 and 15 illustrate a third specific example of conversation(utilizing conversation) that may be held between the robot and the userin order to utilize the value of an item under the control of the memoryutilizing conversation generating section 51. In the illustratedexample, assume that the topic is “the birthday of the object ID1” andthe third topic utilizing method determining rule is selected as methodof utilizing the topic.

Then, the memory utilizing conversation generating section 51 initiallymakes a transition of state from Node 30 to Node 31, where it reads outthe value of the “birthday” of “the object ID1” from the long termmemory 36 (FIG. 5) and computationally determines the constellation,using it as keyword. Then, it generates character string data D1 of thespeech of “Yukiko is a girl of Taurus, isn't she?” by using the obtaineditem of “Taurus” and outputs the speech to the voice synthesizingsection 53 (FIG. 5).

Then, if the memory utilizing conversation generating section 51confirms at Node 31 that the user responds positively, saying “Yes”, onthe basis of the outcome of the voice recognition of the auralrecognition function section 32 (FIG. 5), it makes a transition of statefrom Node 31 to Node 32 and retrieves the constellation data base, using“Taurus” as keyword. As the memory utilizing conversation generatingsection 51 acquires the item of “the day of traffic advertisement”, itoutputs character string data D1 of a speech of, for example, “People ofTaurus are patient aren't they?” to the voice synthesizing section 53.Thereafter, the memory utilizing conversation generating section 51further makes transition of state from Node 32 to Node 33.

If, on the other hand, the memory utilizing conversation generatingsection 51 confirms at Node 31 that user responds negatively, saying“No”, on the basis of the outcome of the voice recognition of the auralrecognition function section 32, it makes a transition of state fromNode 31 to Node 33 and outputs character string data D1 of the speedcorrelated to arc Ac23, the speech of “Sorry” in this example, to thevoice synthesizing section 53.

As the memory utilizing conversation generating section 51 makes atransition of state to Node 33, it notifies the situation judgingsection 52 (FIG. 5) of the fact that a series of speeches is over andsubsequently ends the processing operation for generating this series ofutilizing speeches.

(3) Operation and Effects of this Embodiment

With the above described arrangement, the robot 1 acquires variouspieces of information on the user by way of conversations (acquiringconversations) with the user and stores the acquired information in thelong term memory 36 (FIG. 5) as values of items. Then, the robot 1 holdsa (utilizing) conversation with the user, utilizing the values of itemsstored in the long term memory 36.

Therefore, the robot 1 can hold a conversation with the user on the userhim- or herself and/or one or more than one topics relating to the user.In other words, the robot 1 can hold a conversation customized for theuser and thus the user can feel the process of personalization so thatthe user will improve the affinity he or she feels for the robot 1.

The robot 1 generates a conversation according to the first throughsixth topic generation rules. In other words, the robot 1 generates aconversation within a scope that can be associated with the immediatelypreceding conversation. Therefore, the topic will not abruptly jump inthe course of dialog. Additionally, the robot 1 not only utilizes theinformation per se on the user it acquires but also retrieves data suchas data on horoscope and memorial days according to the above describedfirst through third topic utilizing method determining rules, whileholding the conversation, so that the dialog with the robot 1 can becomemore and more interesting and the scope of variations of conversationcan be enlarged.

Additionally, the robot 1 keeps conversation history in the memoryacquiring conversation generating section 50 and the memory utilizingconversation generating section 51 so that the combination of a topicand a utilizing method is not generated repeatedly in a series ofdialogs with the same user on the basis of the conversation history.Therefore, any speeches that relate to the information acquired by therobot 1 and are correlated to the user will not be output immediatelyafter the acquisition of the information so that the conversation beingheld between the robot 1 and the user is prevented from becoming boring.

Additionally, the robot 1 is adapted to switch from an acquiringconversation to a utilizing conversation or vice versa under the controlof the situation judging section 52 depending on the situation ofacquiring information correlated to the user. Therefore, the robot 1will neither utilize its memory in a situation where no information thatis correlated to the user is available nor try to further acquireinformation in a situation where all information is acquired. In short,the robot 1 is prevented from acting unnaturally.

Additionally, the robot 1 stores the values of items of the user and soon in the long term memory section 36 along with degrees of impressionand is adapted to select a topic on the basis of the stored degree ofimpression of the topic. Therefore, the frequency of appearance of thetopic of an item can be varied as a function of the degree ofimpression. Thus, the robot 1 can highly entertainingly interact withthe user as it talks about what it likes to talk and does not talk aboutwhat it does not like to talk.

With the above-described arrangement, the robot 1 acquires variouspieces of information on the user through conversations with the user.Then, it stores the acquired pieces of information in the long termmemory 36 (FIG. 5) as values for items and holds a conversation with theuser, utilizing the values of items stored in the long term memory 36.Thus, it can hold a conversation customized for the user and thus theuser can feel the process of personalization so that the user willimprove the affinity he or she feels for the robot 1. Such a robot 1 isobviously highly entertaining.

(4) Other Embodiments

While the present invention is described above by way of the embodimenthaving a configuration as shown in FIGS. 1 through 3, the presentinvention is by no means limited thereto. In other words, the presentinvention can provide entertainment robots having a configuration otherthan the one described above and other robots as well as variousapparatus other than robots and various dialog control devices that cancontrol dialogs with a user. The present invention can also be appliedto various pieces of software such as those for television games.

While the degree of impression of each item is determined when the robotacquires the value of the item and will not be updated since in theabove description of the embodiment, the above embodiment may be somodified that the degree of impression can be updated after theacquisition thereof. With this arrangement, the frequency of appearanceof each topic can be modified and the ongoing conversation can morereflect the current situation to make the robot 1 more entertaining.

While the degree of impression of each of various items is determined bycomputing the difference of the parameter values of “affection” held inthe internal condition management section 34 before and after theacquisition of the item as described above by referring to FIG. 4 in thedescription of the embodiment, the present invention is by no meanslimited thereto and the degree of impression of an item may bedetermined by using the parameter values of some other instinctiveelement or emotional element that are held in the internal conditionmanagement section 34. Alternatively, some other technique may be usedto determine the degree of impression of each item.

While the RAM 22 and the nonvolatile memory 24 (FIG. 2) that areinternal memories of the control unit 2 are used as memory means forstoring the values of items of an object in the above description of theembodiment, the present invention is by no means limited thereto andsome other memory mediums may be used. Alternatively some other externalstorage mediums such as external memories may be used for the purpose ofthe invention.

Additionally, three modules are used as conversation generation meansfor selecting a topic associated with the topic used in the immediatelypreceding conversation and generating an acquiring conversation foracquiring the value of the item of the selected topic or a utilizingconversation for utilizing the value of the item of the topic alreadystored in the long term memory 36 as the next conversation and the threemodules include a memory acquiring conversation generating section 50that operates as memory acquiring conversation generation means forgenerating acquiring conversations, a memory utilizing conversationgenerating section 51 that operates as memory utilizing conversationgeneration means for generating utilizing conversations and a situationjudging section 52 that operates as situation judgment means forselecting either the memory acquiring conversation generating section 50or the memory utilizing conversation generating section 51 and havingeither the memory acquiring conversation generating section 50 or thememory utilizing conversation generating section 51, whicheverappropriate, generate the next conversation in the above describedembodiment. However, the present invention is by no means limitedthereto and the conversation generation means may alternatively havesome other configuration.

While a total of six topic generation rules including the first throughsixth topic generation rules are provided to define a method ofgenerating the topic of the next conversation in the above describedembodiment, the present invention is by no means limited thereto andsome other rules may alternatively be used as topic generation rules.The number of such rules is not limited to six.

Similarly, a total of three topic utilizing method determining rulesincluding the first through third topic utilizing method determiningrules are provided to define a method of utilizing the generated topicin the above described embodiment, the present invention is by no meanslimited thereto and some other rules may alternatively be used as topicutilizing method determining rules. The number of such rules is notlimited to three.

As described above in detail, according to the invention there isprovided a dialog control device comprising a memory means for storingvarious pieces of information appendant to an object as valuescorresponding to respective items of the object and a conversationgeneration means for selecting, in response to an item of the objectdefined as topic, another topic relating to the topic used in theimmediately preceding conversation and generating an acquisitionconversation for acquiring the value of the item selected as topic or autilization conversation for utilizing the value of the item in thetopic already stored in the memory means as the next conversation, theconversation generation means being adapted to store the acquired valueacquired by the acquisition conversation as the value of thecorresponding item. Then, with a dialog control device as defined above,the dialog control device with an enhanced level of entertainment canmake a conversation that is customized for the user, or the object, withthe user.

According to the invention, there is also provided a dialog controlmethod comprising a first step of storing various pieces of informationappendant to an object as values corresponding to respective items ofthe object and a second step of selecting, in response to an item of theobject defined as topic, another topic relating to the topic used in theimmediately preceding conversation and generating an acquisitionconversation for acquiring the value of the item selected as topic or autilization conversation for utilizing the value of the item in thealready stored topic as the next conversation, the acquired valueacquired by the acquisition conversation being stored in the second stepas the value of the corresponding item. Then, with a dialog controlmethod as defined above, the dialog control device can make aconversation that is customized for the user, or the object, with theuser.

According to the invention, there is also provided a robot devicecomprising a memory means for storing various pieces of informationappendant to an object as values corresponding to respective items ofthe object and a conversation generation means for selecting, inresponse to an item of the object defined as topic, another topicrelating to the topic used in the immediately preceding conversation andgenerating an acquisition conversation for acquiring the value of theitem selected as topic or a utilization conversation for utilizing thevalue of the item in the topic already stored in the memory means as thenext conversation, the conversation generation means being adapted tostore the acquired value acquired by the acquisition conversation as thevalue of the corresponding item. Then, with a robot device as definedabove, the dialog control device with an enhanced level of entertainmentcan make a conversation that is customized for the user, or the object,with the user.

INDUSTRIAL APPLICABILITY

The present invention is applicable not only to entertainment robots andother robots of various different types but also to devices other thanrobots such as personal computers where dialog features are installed.

1. A dialog control device to customize dialog between a user and arobot comprising: memory means for storing various pieces ofinformation, appendant to an object, as values corresponding torespective items of the object and for storing a degree of impressionrelative to the value, wherein the degree of impression indicateswhether the item is used in future conversations, wherein the variouspieces of information are acquired by the robot and based on a voicerecognition process and visual recognition of a user; and conversationgeneration means for (a) selecting, in response to an item of saidobject defined as a topic about the user, another topic about the userrelating to the topic used in the immediately preceding conversation and(b) generating at least one of (1) an acquisition conversation foracquiring the value of the item, selected as the topic, from the userand (2) a utilization conversation for utilizing the value of the itemin the topic, already stored in said memory means, as the nextconversation based on the degree of impression wherein a frequency of autilized item is varied; said conversation generation means beingadapted to store the acquired value, acquired by said acquisitionconversation, as the value of the corresponding item, wherein the dialogcontrol device makes a conversation with the user that is customized forthe user.
 2. The device according to claim 1, wherein said conversationgeneration means selects any other item of the same object to which thetopic used in said immediately preceding conversation belongs as thenext topic and generates said utilization conversation by utilizing thevalue of the item already stored in said memory means.
 3. The deviceaccording to claim 1, wherein said conversation generation means selectsan item relating to the same object to which the topic used in saidimmediately preceding conversation belongs as the next topic andgenerates said utilization conversation by utilizing the value of theitem already stored in said memory means.
 4. The device according toclaim 1, wherein said conversation generation means selects any of theitems of said object identifiable from the value of the item of thetopic used in said immediately preceding conversation as the next topicand generates said utilization conversation by utilizing the value ofsaid any of the items already stored in said memory means.
 5. The deviceaccording to claim 1, wherein said conversation generation means selectsthe item of said object same as the topic used in said immediatelypreceding conversation as the next topic and generates said utilizationconversation by utilizing the value of the item already stored in saidmemory means.
 6. The device according to claim 1, wherein saidconversation generation means selects the same item of another objecthaving a value same as the value of the item of the topic used in saidimmediately preceding conversation as the next topic and generates saidutilization conversation by utilizing the value of the same item alreadystored in said memory means.
 7. The device according to claim 1, whereinsaid conversation generation means selects an item of another objectrelating to the value of the item of the topic used in said immediatelypreceding conversation as the next topic and generates said utilizationconversation by utilizing the related value already stored in saidmemory means.
 8. The device according to claim 1, wherein saidconversation generation means selects any other item of the same objectto which the topic of said immediately preceding conversation belongs asthe next topic and generates said acquisition conversation in order toacquire the value of said any other item.
 9. The device according toclaim 1, wherein said conversation generation means selects an itemrelating to the same object to which the topic used in said immediatelypreceding conversation belongs as the next topic and generates saidacquisition conversation in order to acquire the value of the relateditem.
 10. The device according to claim 1, wherein said conversationgeneration means selects any of the items of said object identifiablefrom the value of the item of the topic used in said immediatelypreceding conversation as the next topic and generates said acquisitionconversation in order to acquire the value of said any of the items. 11.The device according to claim 1, wherein said conversation generationmeans generates said utilization conversation by utilizing a matter thatcan be acquired on the basis of the value of the item of said selectedtopic.
 12. The device according to claim 1, wherein said conversationgeneration means includes: memory acquisition conversation generationmeans for generating said acquisition conversation; memory utilizationconversation generation means for generating said utilizationconversation; situation judgment means for selecting either said memoryacquisition conversation generation means or said memory utilizationconversation generation means and have said memory acquisitionconversation generation means or said memory utilization conversationgeneration means, whichever selected, generate said next conversation.13. The device according to claim 12, wherein said situation judgmentmeans judges to have either said memory acquisition conversationgeneration means generate said acquisition conversation or said memoryutilization conversation generation means generate said utilizationconversation on the basis of the first extent that is the ratio of thenumber of items whose values are not acquired relative to the totalnumber of items of the dialog partner and the second extent that is theratio of the number of items whose values are acquired relative to thetotal number of items of said dialog partner.
 14. The device accordingto claim 13, wherein said situation judgment means has said memoryacquisition conversation generation means generate said acquisitionconversation when said first extent is greater than said second extentbut has said memory utilization conversation generation means generatesaid utilization conversation when said first extent is smaller thansaid second extent.
 15. The device according to claim 1, wherein saidconversation generation means holds history of the used topics andgenerates said acquisition conversation or said utilization conversationby referring to the history.
 16. The device according to claim 15,wherein said conversation generation means generates said acquisitionconversation or said utilization conversation so as not to use the sametopic in a single dialog with the same dialog partner by referring tothe history.
 17. The device according to claim 1, characterized byfurther comprising: internal condition management means adapted to holdparameters indicating the internal condition and change the values ofthe parameters according to external stimuli; each of said degrees ofimpression representing the difference of the parameter values of thecorresponding parameter held by said internal condition management meansbefore and after the acquisition of the corresponding value.
 18. Adialog control method to customize dialog between a user and a robotcomprising: a first step of storing various pieces of information,appendant to an object, as values corresponding to respective items ofsaid object and for storing a degree of impression relative to thevalue, wherein the degree of impression indicates whether the item isused in future conversations, wherein the various pieces of informationare acquired by the robot and based on a voice recognition process andvisual recognition of a user; a second step of selecting, in response toan item of said object defined as a topic about the user, another topicabout the user relating to the topic used in the immediately precedingconversation; and a third step of generating at least one of (a) anacquisition conversation for acquiring the value of said item selectedas the topic from the user and (b) a utilization conversation forutilizing the value of the item in the already stored topic as the nextconversation based on the degree of impression wherein a frequency of autilized item is varied; the acquired value, acquired by the acquisitionconversation, being stored in the second step as the value of saidcorresponding item, wherein the robot makes a conversation with the userthat is customized for the user.
 19. The method according to claim 18,wherein any other item of the same object to which the topic used insaid immediately preceding conversation belongs is selected as the nexttopic and said utilization conversation is generated by utilizing thevalue of the item already stored in said memory means in said secondstep.
 20. The method according to claim 18, wherein an item relating tothe same object to which the topic used in said immediately precedingconversation belongs is selected as the next topic and said utilizationconversation is generated by utilizing the value of the item alreadystored in said memory means in said second step.
 21. The methodaccording to claim 18, wherein any of the items of said objectidentifiable from the value of the item of the topic used in saidimmediately preceding conversation is selected as the next topic andsaid utilization conversation is generated by utilizing the value ofsaid any of the items already stored in said memory means in said secondstep.
 22. The method according to claim 18, wherein the item of saidobject same as the topic used in said immediately preceding conversationis selected as the next topic and said utilization conversation isgenerated by utilizing the value of the item already stored in saidmemory means in said second step.
 23. The method according to claim 18,wherein the same item of another object having a value same as the valueof the item of the topic used in said immediately preceding conversationis selected as the next topic and said utilization conversation isgenerated by utilizing the value of the same item already stored in saidmemory means in said second step.
 24. The method according to claim 18,wherein an item of another object relating to the value of the item ofthe topic used in said immediately preceding conversation is selected asthe next topic and said utilization conversation is generated byutilizing the related value already stored in said memory means in saidsecond step.
 25. The method according to claim 18, wherein any otheritem of the same object to which the topic of said immediately precedingconversation belongs is selected as the next topic and said acquisitionconversation is generated in order to acquire the value of said anyother item in said second step.
 26. The method according to claim 18,wherein an item relating to the same object to which the topic used insaid immediately preceding conversation belongs is selected as the nexttopic and said acquisition conversation is generated in order to acquirethe value of the related item in said second step.
 27. The methodaccording to claim 18, wherein any of the items of said objectidentifiable from the value of the item of the topic used in saidimmediately preceding conversation is selected as the next topic andsaid acquisition conversation is generated in order to acquire the valueof said any of the items in said second step.
 28. The method accordingto claim 18, wherein said utilization conversation is generated byutilizing a matter that can be acquired on the basis of the value of theitem of said selected topic in said second step.
 29. The methodaccording to claim 18, wherein it is judged to generate either saidacquisition conversation or said utilization conversation on the basisof the first extent that is the ratio of the number of items whosevalues are not acquired relative to the total number of items of thedialog partner and the second extent that is the ratio of the number ofitems whose values are acquired relative to the total number of items ofsaid dialog partner in said second step.
 30. The method according toclaim 29, wherein said acquisition conversation is generated when saidfirst extent is greater than said second extent but said utilizationconversation is generated when said first extent is smaller than saidsecond extent in said second step.
 31. The method according to claim 18,wherein history of the used topics is held and said acquisitionconversation or said utilization conversation is generated by referringto the history in said second step.
 32. The method according to claim31, wherein said acquisition conversation or said utilizationconversation is generated so as not to use the same topic in a singledialog with the same dialog partner by referring to the history in saidsecond step.
 33. The method according to claim 18, wherein parametersindicating the internal condition are held and the values of theparameters are changed according to external stimuli in said first step;each of said degrees of impression representing the difference of theparameter values of the corresponding parameter held by said internalcondition management means before and after the acquisition of thecorresponding value.
 34. A robot device that customizes dialog with auser comprising: memory means for storing various pieces of informationappendant to an object as values corresponding to respective items ofthe object and for storing a degree of impression relative to the value,wherein the degree of impression indicates whether the item is used infuture conversations, wherein the various pieces of information areacquired by the robot and based on a voice recognition process andvisual recognition of the user; and conversation generation means for(a) selecting, in response to an item of said object defined as a topicabout the user, another topic about the user relating to the topic usedin the immediately preceding conversation and (b) generating at leastone of (1) an acquisition conversation for acquiring the value of theitem, selected as the topic, from the user and (2) a utilizationconversation for utilizing the value of the item in the topic, alreadystored in said memory means, as the next conversation based on thedegree of impression wherein a frequency of a utilized item is varied;said conversation generation means being adapted to store the acquiredvalue, acquired by the acquisition conversation, in said memory means asthe value of the corresponding item, wherein the robot makes aconversation with the user that is customized for the user.